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p^ Abstract 

J^ In many applications, it is needed to change the topology of a tensor network 

~^ directly and without approximation. This work will introduce a general scheme 
that satisfies these needs. We will describe the procedure by two examples and 

j^ show its efficiency in terms of memory consumption and speed in various numerical 

^^ experiments. In general, we are going to provide an algorithm to add an edge to 

^^ a tensor network as well as an algorithm to remove an edge unless the resulting 

r network is a connected graph. 
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1 Introduction 
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^ Tensor networks are of interest especially in quantum chemistry (see [I1I2]) but also for 

, ^, general large data sets, they can become of practical use. 

Changing the topology of a tensor network, i.e. representing a tensor given in structure 

►^ A as a tensor in structure B may result in the opportunity to treat originally differently 

^r^ structured tensors equally. 

^^ One main interest is to convert an arbitrary tensor network into a tree structured 

' tensor network which has nice properties in terms of stability and computational effort. 

•^ More about the indicated properties can be found in |3], |1] and |5]. 
(^ We want to consider only conversion from connected graph structured tensors to 

CN connected graph structured tensors where rank 1 edges will be neglected. 



2 Problem description 



> 
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j^ One of the main problems with arbitrary tensor networks is, that they might contain 

cycles. This leads in the representation to be not stable (see j.6]) which results in addi- 
tional conditions that have to be preserved while performing algorithms. With having 
a procedure that stabilizes the representation by changing its structure to a tree struc- 
ture (which is stable, see jTl Lemma 8.6 for the Tucker format and Lemma 11.55 for the 



' ^Stefan . hand s chuhOmis . mpg . de Max-Planck-Institute for Mathematics in the Sciences, Leipzig, 
Germany 



Hierarchical format] and [8, Theorem 3.2]) we can avoid the constraints. For example, 
one could transform the structure into a tree, perform stable computation with it and 
re-transform it back to the original structure. 

Another problem is that for general tensor networks the contraction (i.e. carrying out 
the summations) is hard to perform in terms of the computational cost. If an algorithm 
has to compute the inner product after each iteration, this will become a main part of 
the whole cost of the algorithm. Contracting tree structured tensor networks however, 
has a much smaller complexity that is linear in the dimension of the tensor network. 

Furthermore, it is important for the addition of tensors that are represented in net- 
works, that the formats of all terms have a matching structure. Performing computations 
with differently structured tensor networks is in general harder than performing compu- 
tations with equally structured ones. This even holds for different tree structures which 
should be avoided. See 0] Section 5.2] for an example. 

3 Direct conversion from TC to TT w/o approximation 

Our first example will be the topology change from a ring structure (Tensor Chain or 
TC, see below) to a string structure (Tensor Train or TT, see below). The two topologies 
differ only in one edge and therefore they have a lot in common which we can use for our 
advantage. 

Let d G N>2, "Ri, . . . , rid, '^ii • • • j '"d S N, then we define 

ri,...,rd d 

iiv,id=i M=i 

with 

z;i:{l,...,rrf}x{l,...,ri}^K"i 

Vi ■.{!,..., ri_i} X {1, . . . , rj} -> K"' for i = 2, . . . , d 
as a Tensor Chain representation with representation rank (ri, . . . ,rii), see Figure n] 
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Figure 1: Tensor Chain of order d 



Our goal structure is defined by 

fi,...,fd-i 



V = 

JlvJd-l 



y^ Vlijl) V2iJl,J2) ...0 Vd-liJd-2,jd-l) ^ Vd{jd-l) 



with 



Vl 

Vi 
Vd 



{l,...,fi}^]K'^^ 

{l,...,fi_i}x {!,..., fJ^]IC"> foii = 2,...,d-l 



which is called a Tensor Train representation with representation rank (fi, . . . ,fd-i) S 
N and visualized in Figure^ This is a special case of the Tensor Chain format (see 

|9|)- 
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Figure 2: Tensor Train of order d 



To be able to use the given ring structure of the Tensor Chain, we will convert it to 
the simplest possible order d tree, which is the Tensor Train. We successively move one 
specific edge of the ring further and further to the edge that is at the center of the ring 
without this specific moving edge. The following part visualizes this scheme. 

In the following description, we are using the singular value decomposition (SVD) to 
decompose a matrix. We could also utilize other decompositions, like the QR decompo- 
sition, but they have the same computational complexity as the SVD. A main advantage 
of the SVD is, that it provides a best rank k approximation for matrices which we want 
to use later in approximated results. 

1st step 

We define 

ri 
Vl,2{jd,J2) ■■= ^Vi{jd,jl)(E)V2{jl,J2) 

ii=i 



and interpret U1.2 as ni x 77-2 • r^^ • r2 matrix on which we apply the SVD to obtain 



Vl,2{jd,J2) = ^ v'liji) (g) V2{jl,J2,jd), 
ii=l 

where fi < rii is the full SVD rank. Consequently, 

fi r2,-,rd 

ii=ii2v,id=i 
whose schematic representation is Figure [3] 




j2,---,Jrii 



m-2 



Jfd] , 2)--- iJd-2 



'rii+2 



VdiJd-lJd) 




rrii-i 




rrii+4 



Jr^ 



ni- 



j[i 



m+i 




'ufd 



Jr4 



ni 




rrfi+i 



Figure 3: Structure after the 1st step 



2nd step 

Analogous to step 1, we define 



rd-i 



Vd-l,dUd-2,jd) ■■= X Vd-l{jd-2,jd-l)®Vd{id~l,3d) 
jd-l=l 

and interpret Vd-i,d{jd-2^jd) as n^-i x n^ ■ rd-2 ■ fd matrix, of which we compute the 
SVD, in order to get 

fd-i 

Vd-l,dUd-2,jd) = ^ v'^_^{jd-2,jd-l,jd)®v'^{jd-l) 
3d-\='^ 

where again fd-i is the full SVD rank. The result is 

n,'rd_i r2,..;rd-2,rd 
V= ^ ^ v'i{h)®v'2{h,3dih)®Vz{J2,h)® ■■■®Vd-2{jd-?,^3d-2) 

ii jd-i=ii2vjd-2jd=i 

® Vd-l{jd-2,jd-l,jd) ® v'a{jd-l) 



as visualized in Figure |4] 
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Figure 4: Structure after the 2nd step 



Penultimate step 

We apply the above written scheme successively, we end up in a situation that is equiv- 
alent to Figure [5} 



qy^^-Q) 



j2,---,J[dl_ 



[11-2 



© .-. © 



J[il^2^---'Jd-2 



'[11+2 




Figure 5: Structure before the penultimate step 



The penultimate step is to apply the SVD to 

7^1- 



E ^[f]+i(j[f],i[f]+i)»^[|]+2(irfi+i'j'rfi+2>j<^ 



SVD 



I 3 



7fl+^ 



and obtain 



'■^'■••'7fl-i'7^1+^'-''''-^ li'"'"' 






.,jd-i=iirdi Jd=i 



^ w'l(jl) l» V2{ji,J2) . . . O ^['|]_i (i[d^-2'i[f]- 



with the corresponding Figure |6j 
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Figure 6: Structure after the penultimate step 



Remark 3.1. Formally speaking, this structure is already the Tensor Train format since 
we can interpret edge jd and jvdi as together as one edge with multiplied ranks. 
This situation however, can he improved by performing one additional SVD to combine 
the two edges to be able to obtain the real rank of the center edge. 



Final step 

In the last step, we perform a singular value decomposition of 



E ^[fi (jrfi-rinvjrf) ^-||]+i {^i-\.j\iy,^3d 



ird"i ,id=i 



and obtain analogously to the previous step a structure that is visualized in Figure [7] 
The distinction of ' and " is important since " means that this node has been changed 
by two SVDs whereas ' stands for one SVD. On the other hand, ~ indicates that a double 
edge has been united. 
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Figure 7: Completely converted structure 



Ranks 

If we consider the new ranks fi, . . . , fd-i, we have to look at the dimension of the matrices 
that we decompose via the SVD. We have 



n = "-1 

fi = min(ni • fi_i, nj+i • r^ • rj+i) < min(n*, n^+i • r^ • rj+i) 



for i = 2, 



rd_i = Ud 



fi = min(nj+i • fj+i, nj • r^ ■ n^i) < min(n \ rii ■ r^ ■ rj_i) for i = d - 2, . . . , 

^[fl =min(^[f] •^[|]-i>"[|]+i-^[|]+i)<min(nril,n2.r2), 
which is summarized in Figure [8| 
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Figure 8: Final rank overview 



Remark 3.2. With this approach the upper bounds for the ranks are the full TT-ranks 
(see /lOl and compare with (II]) - ^), but also available TC-ranks influence the resulting 
representation rank. 

Theorem 3.3. The overall computational cost for the conversion is in 

O {{d - 2) • nV^ + nV^) , 

so it is linear in d where r := niax(ri, . . . , r^-i). 

Proof. Due to (II| - (|4]), we have 



fi <n ■ r^- ri <n ■ r \/i £ {1, . . . , d — 1} \ { 



}• 



Consequently, the matrices, that we have to decompose with the SVD have at most the 
size 

} 



n ■ fi X n ■ r^ ■ r ViG{l,...,(i— 1}\{ 
except for the final step. There, the matrix has at most the size 

due to ([5|, which finishes the proof. 



D 



Remark 3.4. Steps 1, 3, . . . , di and 2, 4, . . . , ^2 are independent of each other and there- 
fore parallelizable where 



di := 



d-3 ifd = mod 2, 
d — 2 otherwise 



and 



do:-- 



d-2 ifd = mod 2, 
d — 3 otherwise. 



We can easily extend this scheme to more complex structures which we will do in 
Section |4] by converting a 2d grid structured tensor into a string structured tensor. 

3.1 Numerical example 

All numerical experiments in this paper have been done with |11) with the following 
setup: 

• the function values have are generated with a pseudo-random number generator 

• each direction has 10 entries, i.e. ni = . . . , n^ = 10 

• the representation rank of the tensor chain tensor is (ri, . . . , r^) = (6, . . . , 6) 



d 


CPU-time 


Avg. rank 


Max. rank 


4 


0.01s 


18.67 


36 


10 


1557.59s 


188.44 


360 


100 


1845.43s 


344.40 


360 


1000 


3850.65s 


358.45 


360 



Table 1: Exact TC to TT conversion 

In practice, it is often sufficient to convert a format only approximately instead of a 
non-approximated conversion. Our approach can be easily changed to an approximated 
conversion by using the SVD only up to a certain accuracy. We will demonstrate this by 
simple computations with an allowed SVD error of 10"^'' 



d 


CPU-time 


Avg. rank 


Max. rank 


rel. error 


4 


0.01s 


18.67 


36 


1.49 • 10~« 


10 


2.93s 


30.22 


36 


1.61-10-7 


100 


56.6s 


35.47 


36 


7.12-10-7 


1000 


551s 


35.95 


36 


1.54-10-*^ 


10000 


6042s 


35.99 


36 


1.83 - 10-5 



Table 2: Approximated TC to TT conversion 

Remark 3.5. We do not need to hold the whole tensor in the RAM since the conversion 
acts only locally on the two involved edges. This reduces the practical memory consump- 
tion to a very small fraction of the theoretical consumption (when storing the whole tensor 
in the RAM). Especially if we increase the accuracy of the singular value decomposition 
by increasing the rank, this locality-advantage plays an important role. 



4 Converting PEPS to TT w/o approximation 

In Section |3j the topology changed only slightly as we removed just one edge from the 
graph to obtain a tree. The method that has been used there can be also used for more 
complicated structures such as 2d grids which we want to explain in this section. 

We are going to convert a PEPS (projected entangled pair state; see [12] for applica- 
tions) structured tensor into a tree structured tensor in the TT-format. In our framework 
of arbitrary tensor representations, a PEPS- Tensor of order 16 has the following formula: 

ri,--.,r24 
V= ^ l'l(i4,il) ® V2iJl,J5,J2) ® V3{J2,hj3) ® W4(j3, jV)® 
jl,-,J24 = l 

VbiJiJsJii) O ve{J8,J5,Ji2,J9) ^ V7{J9,h,Ji3,Jio) ® vsijiojr, ju)^ 

VgiJu, J18, jib) ^ Vl0{jl5, J12, J19, jw) ^Vll{jl6,jl3,J20,jn) ^ Vuijll, ju, J2l) 
^'l3(il8,i22) ^ Vu{J22,jl9,J23) ® ^isfe, J20, i24) ^ ^^16(j24, i2l), 

see Figure |9] for the visualization. The Motivation for this conversion is due to the 
fact that the complexity of contracting a PEPS tensor is very high and the optimization 
procedure is not stable (see [T^ for an approximated contraction scheme). Tree structured 
tensors on the other hand are easy to contract and stable. 

Each tree with p vertices has p — 1 edges such that it is reasonable to choose the sim- 
plest tree structure, which is a string, as the destination structure. This is no restriction 
of the method, we just chose the string structure only for visual reasons. 

For the sake of simplified notations, we set ni = . . . = n^ =: n S N, so all our vector 
spaces Vjj, have the same dimension n. 



10 



GmKE) 




Figure 9: PEPS 



We want to visualize the scheme that we introduced in Section |3] by looking at the 
upper left corner of the PEPS tensor. Figure 10(a) displays the initial situation. 
The first step, that we want to perform is moving the edge J4 to the left. 






and we get the structure 10(b) Hereafter, we perform 



rg 



rg 



X] ^5(J4, J8, ill) ® ve{J5,J8,kJi2) = X ^^ikJn) <^ v'f^{k,k,k,k,j 



13 J 



which is shown in Figure 10(c) The next step could be to move those two edges J4 and 



j5 both further to the left, but this would increase the complexity of the formulas as well 
as of the schematic drawings. Additionally, it might be the case that the product of the 



moved edges ranks is to high (see Remark 3.1 ). 



So, we want to combine J4 and j^ into a new J5 and we can do this by one SVD: 



ri,rh 



X ^2(J1;J2,J4,J5)(»W6(J4,J5,J8,J9,J12) = "^ V2{k^ k, k) ® V&ik, k, h, k2) 
J4,J5 = l i5 = l 

such that we get a structure as of Figure |10(d)[ 
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(b) State after 1st step 
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(c) State after 2nd step 
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(d) State after 3rd step (e) State after 4th step 

Figure 10: Iteration series 1 in details 



Afterwards, we can proceed as before (see Figure 10(e) ). If we apply this procedure 



until we have eliminated also edges j's and jg with edge jj left, we get a structure as in 
Figure 11 
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Figure 11: PEPS series 1 



Applying this scheme also on edges (jiSj jig, J20) we first get the structure of Figure 



12 and afterwards the structure of Figure 13 if we eliminate edges (ji4,ii3,ii2) 




Figure 12: PEPS series 2 
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The elimination processes of (j4,i5,j6) a-nd of {ji8,ji9,J2o) do not affect each other 
such that we can state the rank distribution independently: 






r2 = n 
fg = n 
vq = n 

f3=n 

fio =n 

r-j = n 

r22 = n 
fi5 =n 
fi9 =n 

^23 =n 
he = n 
r20 = n 

f24 = n 
fi7 = n 
r2i = n 



min(l, r2 • r4 • r^) = n 
min(rii,r4 • rs • rg • ri2) 
min(fi ■ r2,rs ■ rg ■ ri2) 



n ■ min(n • r2, rg • rg • ri2j 



min(ri, r3 • r^ ■ vq) = n ■ niin(n, r^ ■ r^, ■ rg) 
min(f8 •ri2,r5 ■ vq ■ r^ • ris) 
min(f2 -rsjfg • no • ns) 

min(f2,f6 • rj) 

min(fg -risjfe • ry • ru) 

min(f3,fio -tm) 

min(l,ri8 • rig • r23) = n 

min(rii,ri2 ■ ne • ris -rig) 

min(f22 • r23, ri2 • ns • rie) = n • min(n • r23, ri2 • ri5 ■ rw) 

min(f22, rig • r2o • r24) = n ■ min(n, fig • r2o • r24) 
min(ri2 • ri5, ^13 • rn ■ ng • r2o) 
min(f23 •r24,ri3 ■ fiQ ■ m) 

min(f23,f2o •r2i) 
min(ri3 •fi6,ri4 • f2o • r2i) 
min(f24,ri4 • fn) 
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Figure 13: PEPS series 3 



r 10 = n • min(f7, fg • ris • tm) 
ri7 = n • min(f2i, ri3 • ru ■ ne) 
ri3 = n • min(f9- rio, ne- rn) 

rg =n- min(rio, fs ■ ru ■ fu) 
r 16 = n • min(r 17, ru ■ fu ■ ns) 
ri2 = n • min(f8- rg, fig- rig) 

rg =n-min(r9,rii -7^12) 

ri5 =n-min(ri6,rii -7=12) 

rii = n • min(ri5, rg) 

Resulting in the tree structure which had to be estabhshed. 

Remark 4.1. Series 1 and series 2 are parallelizahle without any restriction since they 
do not have a vertex in common. Series 3 can he performed at the same time as series 
1 and 2 hut one has to he careful with overlapping cycle elimination series since it might 
he possihle that vq is changed hy two processes at the same time, for instance. This can 
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be worked around by adding simple synchronizers. Note that there is at most one edge of 
the edges in common of two processes that may be changed simultaneously. 

Performing the conversion in parallel may lead to different ranks in the ranks that 
are adjusted more than once. 

Remark 4.2. The order of the series is not unique. One can choose any other series 
that produces a string like tree. 

Since we are removing in general d — Isfd + 1 = (V" — 1)^ edges from the graph, we 
get a complexity of the whole algorithm that is quadratic in d (with some factor < 1). 

5 Direct conversion from TT to TC w/o approximation 

For some applications it is needed to destroy the tree topology of a tensor network in 
favor of a more complex structure. For example if one has converted a cycle structured 
tensor network into a tree to perform stable algorithms and after the computation the 
original structure is needed again. 

The reader is reminded of the definition of the Tensor Train format that has been 
introduced in Section |3] 

We want to convert the Tensor Train into a cyclic structured tensor (Tensor Chain). 
In general, every Tensor Train is already a Tensor Chain, since there is a rank one 
edge between v\ and v^ on every Tensor Train. Our objective here is to get a balanced 
distribution of the ranks in the Tensor Chain and to obtain that, we have to perform a 
procedure that successively moves an artificially inserted edge to the start v\ and the end 
Vd of the train. In practice however, this leads to several problems that are inspected in 
Section 15.11 

This procedure depends on the singular value decomposition (SVD) and we want to 
mark a node that has been change by the SVD once with ' and a node that has been 
changed twice with ". 

1st step 

Our first step will be to introduce an artificial edge between node vvd-\ and f rdi , ^ which 



we want to name j^ (see Figure 14 for the visualization). 
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Figure 14: Artificially added edge j^ 



We choose Vd and frd-i such that r^ ■ frdi > rrdi and define the mapping 



T 



m-'m 



[•,•] : {l,...,fpd^} X {l,...,rd} -^ {l,...,rd-f^di^} 

a, 6 I— )■ a + frd-| • 6, 

so [•, •] is a bijective map to assign a 2-tuple to a natural number. Consequently, we have 

7^1 



E v^l][j\l]-vJ\l])^^y\l]+i{j\l]^J\l]^ 



irdi=i 



3 



^ "Til I7ii-i' [^ii'^^JJ ^^m+i [[^\iv^'\ '7fi+i 
^pd^jd=i 



2nd step 

In this step, we want to move the edge j^ from node t'rd-i , -^ to Wrd-i ,2 and as written 
before, we will do this with a single SVD. 
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3rd step 

Edge jd has to be moved to node f rdi-j^ and this will be done analogously to the second 
step. 
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Figure 15: Situation after the 3rd step 
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Remark 5.1. 5iep 2 anc? 3 are independent of each other and can be performed in 
parallel. 



Final step 

After moving the edge successively further towards v^ and vi, we get the situation that 
is visuahzed in Figure [16] The last step in the conversion is to move edge jd from node 
V2 to node vi with the described procedure. 
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Figure 16: Situation before the final step 



So in formulas, one SVD is performed to make the edge shift: 



E Vl{jl)0v'2Ul,J2,jd) 



SVD 



ri 



3d,32 






3d,32 



which is resulting in the structure that we wanted to obtain (see Figure 17) 
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Figure 17: Situation after the final step 

Ranks 

After we have chosen the ranks r^ and rrdi , we update all remaining d — 2 ranks and 
get the following upper bounds 

'd' 



fi = min(nj • rj_i • r^, nj+i • fj+i) < n ■ r ■ ra for z = 1, ...,-- 1 (6) 

ri = mm{ni+i-ri+i-rd,ni-ri-.i) <n-r -ra for z = - +l,...,d-l. (7) 
Theorem 5.2. The computational cost of the described scheme is in 

O {{d - 2) ■ n^r^rfj 
so it is again linear in d with r := niax(ri, . . . , r^-i). 

Proof. Follows directly from Equations rol) and ([7| since these equations determine the 
upper bound for the matrix sizes. D 

5.1 Problems 

The main problem has its roots in the first step where an artificial edge is introduced into 
the graph structure. There we do not a priori know what the best rank splitting is and 
we also do not know which is the best assignment for the [•, •] function. If we can solve 
these problems, we are - for example - able to convert a tensor chain formatted tensor 
into a tensor train formatted tensor and back without different ranks for the tensor chain 
tensor in before the conversion and after the back-conversion. 

5.2 Numerical example 



We have the same setup as in 3.1 (except that we are converting a Tensor Train rep- 
resentation into a Tensor Chain representation) and obtain the following results with 
approximated SVD with a cutoff at 10~^°: 
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d 


CPU-time 


Avg. rank 


Max. Rank 


Rel. error 


4 


0.01s 


7.25 


12 


1.49 • 10"*^ 


10 


0.09s 


10.1 


12 


4.21 • 10-^ 


100 


1.75s 


11.81 


12 


1.05-10-8 


1000 


18.5s 


11.98 


12 


1.17-10-7 


10000 


189s 


12 


12 


2.33 - 10-^7 



Table 3: Approximated TT to TC conversion 



To illustrate the problem that has been described in Section 5.1 , we will run a second 
experiment: first, we will transform a tensor chain tensor into a tensor train tensor and 
then, we will re-transform it back to the original chain format. In this experiment, we 



also have the same setup as in 3.1 (initial TC representation rank is (6, . . . , 6)). No SVD 
approximation is considered. 

d Avg. converted TT-rank Avg. re-converted TC-rank 



40 

244 

648.57 



55 
224 
1815 



Table 4: Exact TC to TT to TC conversion 



In the previous computation, we used the full SVD ranks such that we did not benefit 
from approximated ranks. So we are going to change the algorithm to not use the full SVD 
rank, but an approximated SVD rank with an accuracy of 10-^^ for each singular value 
decomposition for both conversions. The error is the relative error with respect to the 
initial representation. The initial TC representation rank is also (ri, . . . , r^) = (6, . . . , 6). 



d 


Avg. converted TT-rank 


Rel. 


error 


Avg. re-converted TC-rank 


Rel. 


error 


4 


18.67 


4.47 - 10-8 


33 


5.96-10-8 


6 


25.6 


3.33 


10-8 


26 


1.86 


10-8 


8 


28.57 


2.58 


10-8 


28.5 


3.65 


10-8 


10 


30.22 


2.98 


10-8 


30 


1.05 


10-8 


12 


31.27 


4.47 


10-8 


31 


5.58 


10-8 


20 


33.26 


3.33 


10-8 


33 


4.94 


10-8 


30 


34.21 


3.64 


10-8 


34 


3.65 


10-8 



Table 5: Approximated TC to TT to TC conversion 

6 Error estimate 

While shifting an edge, we can introduce an error by omitting small singular of the SVD's 
result. Doing that, we can represent matrix A by an approximated matrix A where we 
can control the error ||^ — ^|| with these singular values. The influence on the whole 
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tensor network representation has to be investigated: We consider the change that is 
made in the second step of the TC to TT conversion of Section |3} We define 

V ■='^ ^ V(jl) 8) V2{jl,J2,jd) ^ V2,{J2,h) ® ■■■® Vd{jd-lJd) 

ii=ii2v,id=i 
and 



ri,rd-i r2,...,rd_2,r-d 
^ ■= ^ ^ '^1 (il ) ^ V2 ijl , 32 , jd) ® Vs (j2 ,h)(E> ... 

ii jd-i=ii2,---.id-2jd=i 

<^ Vd-2{jd-3jd-2) «> Vd-l{Jd-2,jd~l,jd) «> v'dijd-l), 

such that we have to estimate ||f — v\\. To shorten the notation, we introduce 

fi r2,.--,rd-3 
v{jd,jd-2) := X] X] ^l(-^'l) ^ V2{jl,J2,jd) ® V3{J2,J3) ® • • • Vd-2{jd-3, jd-2) 

ii=ii2vjd-3=i 
and 

■'"d-i ?d-i 

B{jd,jd-2) ■■= ^ Wd_i(id_2,jd--l)®?Jd(id-l,jd) - X] ^'d-l{Jd-2,3d-l,3d)®v'd{j 

jd-i=^ id-i=i 

This leads into the following estimate 

^d,'"d-2 



d-lj 



m — V 



^ *(jrf, jd_2) ® B{jd, Jd-2) 

idjd-2=l 
'■d,fd-2 

< ^ \\v{jdJd-2)\\-\\B{jd,jd^2 

jd,id-2=i 



^•d.^'d-2 



'^d,''d-2 



< ( J2 \\'^(^d,jd-2) 

i.idjd-2=i 



fd,''d-2 



kidJd-2 = l 



X] \\B{jd,3d-2) 

\jd,jd-2 = i 

■ \\A-A\\ 



with the help of the triangle inequality and the Cauchy-Schwarz-inequality (in that or- 
der) . This gives us a precise estimate on when we are allowed to cut off singular values 
while still maintaining a certain error bound for \\v — v\\. 

This error estimate can be easily generalized to other tensor representations. 
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7 Alternative approaches 

The proposed algorithm is of course not the only way to convert an arbitrary tensor 
network into a tensor tree network. For example, one could also evaluate the tensor 
network to obtain the full tensor and perform the Vidal decomposition (see |14| I15| ) in 
order to obtain a tensor in the Tensor Train format. Another possibility is to decompose 
the full tensor with a high order SVD (HOSVD, see |16| ) into a hierarchically formated 
tensor (see [4|). Evaluating the full tensor for large d however is in general not feasible 
due to the amount of storage and computational effort that is needed. 

Another general approach is to fix the resulting format and use approximation al- 
gorithms such as ALS, DMRG (both are non linear block Gauss-Seidel methods, see 
|8]). This however is no direct conversion, but an approximation that has certain con- 
vergence rates. The advantage there is that this approach allows us to use general tensor 
representations without being restricted to tensor networks. 

References 



[1 

[2] 
[3] 
[4 
[5 
[6 

[7 
[8 

[9 



V. Murg, F. Verstraete, O. Legeza, and R. M. Noack. Simulating strongly correlated 
quantum systems with tree tensor networks. Phys. Rev. B, 82:205105, 2010. doiT] 



|10 ■ 1 103/PhysRevB . 82 . 205105 



L. Wang, I. Pizorn, and F. Verstraete. Monte Carlo simulation with tensor network 



states. Phys. Rev. B, 83:134421, Apr 2011. doi : 10 . 1 103/PhysRevB . 83 . 134421 



S. Holtz, T. Rohwedder, and R. Schneider. The Alternating Linear Scheme for 
Tensor Optimisation in the TT Format. DFG SPP 1324 Preprint, 71, 2010. 

W. Hackbusch and S. Kiihn. A New Scheme for the Tensor Representation. J. 



Fourier Anal. AppL, 15:706-722, 2009. |doi : 10 . 1007/s00041-009-9094-9 



I. V. Oseledets. Tensor-Train Decomposition. SI AM J. Sci. Comput., 33(5):2295- 
2317, 2011. |doi : 10 .1137/0907522861 



J. M. Landsberg, Y. Qi, and K. Ye. On the geometry of tensor network states. 



arXiv: 1105.4449 [math.AGJ, 2011. arXiv: 1105 .4449 



Wolfgang Hackbusch. Tensor Spaces and Numerical Tensor Calculus. Springer, 
2012. Idoi : 10 . 1007/978-3-642-28027-6 



M. Espig, W. Hackbusch, S. Handschuh, and R. Schneider. Optimization Problems 



in Contracted Tensor Networks. MIS Preprint, 66, 2011. Available from: http: 
|//www ■ mis ■ mpg . de/publicat ions/preprint s/2011/2011 -66 . html 



B. N. Khoromskij. 0(dlog A^)-Quantics Approximation of N — d Tensors in High- 



Dimensional Numerical Modeling. Const. Approx., 34:1-24, 2010. |doi: 10. 1007/ 
[500365-011-9131-1, 



23 



[10] I. V. Oseledets. Compact matrix form of the d-dimensional tensor decomposition. 
Preprint, 09-01, 2009. Available from: http : //spring . inm . ras . ru/osel/?p=15 . 

[11] M. Espig, M. Schuster, A. Killaitis, N. Waldren, P. Wahnert, S. Handschuh, and 
H. Auer. TensorCalculus library, 2008-2012. Available from: http : //gitorious . _ 



org/tensorcal cuius 



[12] V. Murg, F. Verstraete, and J. I. Cirac. Exploring frustrated spin-systems using 
Projected Entangled Pair States (PEPS). arXiv:0901.2019vl [cond-mat.str-el], 2009. 
[arXiv:0901.2019vn 

[13] T. Huckle, K. Waldherr, and T. Schulte-Herbriiggen. Computations in Quantum 
Tensor Networks. Preprint, 2010. Available from: http : //www5 . in . turn . de/pub/| 
[CompQuaiit Tens orNetwork . pdf ; 

[14] I. V. Oseledets and E. E. Tyrtyshnikov. Breaking the Curse of Dimensionality, Or 
How to Use SVD in Many Dimensions. SIAM J. Sci. Comput., 31(5):3744-3759, 
2009. doi : DPI : 10 . 1137/090748330, 

[15] Guifre Vidal. Efficient Classical Simulation of Slightly Entangled Quantum Compu- 
tations. Phys. Rev. Lett, 91:147902, 2003. do i : 10 . 1103/PhysRevLett . 91 . 147902, 

[16] L. De Lathauwer, B. De Moor, and J. Vandewalle. A Multilinear Singular Value 
Decomposition. SIAM J. Matrix Anal. AppL, 21(4):1253-1278, 2000. , doi: 10. 11377] 
1508954798963056551 



24 



